Automated testing is the process of repetitively and iteratively running through common user or use scenarios in an application to effectively test the known features, boundary conditions, expected application responses and underlying software code paths of the application. Technology to automatically replay user actions to simulate interaction with the application removes the requirement of having a human operator to run though the application repeatedly.
Automated testing can be performed on many types of software from imbedded systems to web-browser-based applications. Automated testing traditionally captures the actions of a user interacting with the user interface of an application through a recording process. The software monitors and recognizes user actions such as button clicks, mouse movements, entering text, and using navigation keys (such as the commonly used <TAB>, <ALT>, <ESC> keys). These actions are then “replayed” by the automated testing software. And as user interactions are replayed systematically, the automated testing software captures any changes in the way the application responds (such as unexpected graphical representations, inconsistent data returned or changes in workflow) and reports them to the automated test software user as possible defects in the application being tested.
To be able to replay a user's interaction with an application over and over again, the automated testing software must keep a script of the user actions. This script describes which actions a user has taken (button clicks, text entries, mouse movements, etc.) as well as the graphical interface objects the user has interacted with. A graphical interface object might be a textbox that accepts text entry, a button that can be clicked, a hyperlink in a web browser that directs the application to a new page, a dropdown list of values, or an application menu command. The script describes both what action was performed by the user and which object the action was performed on. Thus, when the script is “replayed”, the automated testing software can reconstruct the user's actions as closely as possible.
Scripts are usually written automatically by the test recorder in a familiar contemporary programming language such as Visual Basic, Java, C++, or JavaScript. This resulting program code is exposed to the user of the automated testing system, providing a programmatic environment that the user of the automated testing system can take advantage of to add additional programmatic logic. It is not uncommon for automated testing software to extend the underlying programming language to incorporate commands to identify and manipulate graphical interface objects as well as mimic user interactions. The recorder uses these language extensions when automatically writing the script code during a recording session.
But problems exist with this current approach. To comprehensively test an application, a number of these scripts are created to test common features, functionality, logic paths, and programmatic logic of the application. Because these scripts reference graphical interface objects that existed while the original recording took place, they are very susceptible to changes in the application. That is, if a graphical interface object has changed (been renamed, replaced by a different type, a new object has been added, or workflow has changed), the references to the object in the original script may not work. Additionally, scripts are susceptible to changes in the underlying data structure of the application because they are only written at the user interface level (e.g. what UI objects have been interacted with).
If, for example, a UI object receives some data that will be stored in a database, and the definition of the type of data that can be stored in the database changes, the script will have no knowledge of this. This leads to entry of erroneous data in the application and potentially incongruous results during testing. The script may not be wrong at the UI level, but its lack of understanding of the underlying application effectively invalidates its accuracy. The only solution to such a set of problems is to replay all scripts that have been recorded and track down any obvious problems (scripts failing because a UI object that is referenced does not exist anymore) or indirect problems (underlying requirements for data changing). This turns out to be a very time consuming process for the automated test software user. For large software systems, the number of scripts required to comprehensively test an application can be very large. With a very large number of test scripts, maintenance of the tests themselves can be a complex and sizable task. Many times, maintenance of test scripts is the largest component of time involved in implementing automation.
Although present devices are functional, they are not sufficiently accurate or otherwise satisfactory. Accordingly, a system and method are needed to address the shortfalls of present technology and to provide other new and innovative features.